Search for: All records

This paper presents ICAT,1 an evaluation framework for measuring coverage of diverse factual information in long-form text generation. ICAT breaks down a long output text into a list of atomic claims and not only verifies each claim through retrieval from a (reliable) knowledge source, but also computes the alignment between the atomic factual claims and various aspects expected to be presented in the output. We study three implementations of the ICAT framework, each with a different assumption on the availability of aspects and alignment method. By adopting data from the diversification task in the TREC Web Track and the ClueWeb corpus, we evaluate the ICAT framework. We demonstrate strong correlation with human judgments and provide comprehensive evaluation across multiple state-of-the-art LLMs. Our framework further offers interpretable and fine-grained analysis of diversity and coverage. Its modular design allows for easy adaptation to different domains and datasets, making it a valuable tool for evaluating the qualitative aspects of long-form responses produced by LLMs. 

The cooperative modification of spontaneous radiative decay exemplifies a many-emitter effect in quantum optics. So far, its experimental realizations have relied on interactions mediated by rapidly escaping photons, which do not play an active role in the emitter dynamics. Here we use a platform of ultracold atoms in a one-dimensional optical lattice geometry to explore cooperative non-Markovian dynamics of synthetic quantum emitters that decay by radiating slow atomic matter waves. By preparing and manipulating arrays of emitters hosting weakly and strongly interacting many-body phases of excitations, we demonstrate directional collective emission and study the interplay between retardation and super- and subradiant dynamics. Moreover, we directly observe the spontaneous buildup of coherence among emitters. Our results on collective radiative dynamics establish ultracold matter waves as a versatile tool for studying many-body quantum optics in spatially extended and ordered systems. 

Context.Type Ia supernovae (SNe Ia) are a key probe in modern cosmology, as they can be used to measure luminosity distances at gigaparsec scales. Models of their light curves are used to project heterogeneous observed data onto a common basis for analysis. Aims.The SALT model currently used for SN Ia cosmology describes SNe as having two sources of variability, accounted for by a color parameterc, and a “stretch” parameterx₁. We extend the model to include an additional parameter we labelx₂, to investigate the cosmological impact of currently unaddressed light-curve variability. Methods.We constructed a new SALT model, that we dub “SALT3+”. This model was trained by an improved version of theSALTshakercode, using training data combining a selection of the second data release of cosmological SNe Ia from the Zwicky Transient Facility and the existing SALT3 training compilation. Results.We find additional, coherent variability in supernova light curves beyond SALT3. Most of this variation can be described as phase-dependent variation ing − randr − icolor curves, correlated with a boost in the height of the secondary maximum ini-band. These behaviors correlate with spectral differences, particularly in line velocity. We find that fits with the existing SALT3 model tend to address this excess variation with the color parameter, leading to less informative measurements of supernova color. We find that neglecting the new parameter in light-curve fits leads to a trend in Hubble residuals withx₂of 0.039 ± 0.005 mag, representing a potential systematic uncertainty. However, we find no evidence of a bias in current cosmological measurements. Conclusions.We conclude that extended SN Ia light-curve models promise mild improvement in the accuracy of color measurements, and corresponding cosmological precision. However, models with more parameters are unlikely to substantially affect current cosmological results.